Efficient BackProp
Identifieur interne : 002240 ( Main/Exploration ); précédent : 002239; suivant : 002241Efficient BackProp
Auteurs : Yann Lecun [États-Unis] ; Leon Bottou [États-Unis] ; B. Orr [États-Unis] ; -Robert Müller [Allemagne]Source :
- Lecture Notes in Computer Science [ 0302-9743 ] ; 1998.
Abstract
Abstract: The convergence of back-propagation learning is analyzed so as to explain common phenomenon observedb y practitioners. Many undesirable behaviors of backprop can be avoided with tricks that are rarely exposedin serious technical publications. This paper gives some of those tricks, ando.ers explanations of why they work. Many authors have suggested that second-order optimization methods are advantageous for neural net training. It is shown that most “classical” second-order methods are impractical for large neural networks. A few methods are proposed that do not have these limitations.
Url:
DOI: 10.1007/3-540-49430-8_2
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Istex, to step Corpus: 001C12
- to stream Istex, to step Curation: 001A98
- to stream Istex, to step Checkpoint: 001744
- to stream Main, to step Merge: 002357
- to stream Main, to step Curation: 002240
Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title xml:lang="en">Efficient BackProp</title><author><name sortKey="Lecun, Yann" sort="Lecun, Yann" uniqKey="Lecun Y" first="Yann" last="Lecun">Yann Lecun</name></author><author><name sortKey="Bottou, Leon" sort="Bottou, Leon" uniqKey="Bottou L" first="Leon" last="Bottou">Leon Bottou</name></author><author><name sortKey="Orr, B" sort="Orr, B" uniqKey="Orr B" first="B." last="Orr">B. Orr</name></author><author><name sortKey="Muller, Robert" sort="Muller, Robert" uniqKey="Muller " first="-Robert" last="Müller">-Robert Müller</name></author></titleStmt><publicationStmt><idno type="wicri:source">ISTEX</idno><idno type="RBID">ISTEX:ACCB9E0CBDBB8AEDBAF9A8372F13515E36000C20</idno><date when="1998" year="1998">1998</date><idno type="doi">10.1007/3-540-49430-8_2</idno><idno type="url">https://api.istex.fr/document/ACCB9E0CBDBB8AEDBAF9A8372F13515E36000C20/fulltext/pdf</idno><idno type="wicri:Area/Istex/Corpus">001C12</idno><idno type="wicri:Area/Istex/Curation">001A98</idno><idno type="wicri:Area/Istex/Checkpoint">001744</idno><idno type="wicri:Area/Main/Merge">002357</idno><idno type="wicri:Area/Main/Curation">002240</idno><idno type="wicri:Area/Main/Exploration">002240</idno></publicationStmt><sourceDesc><biblStruct><analytic><title level="a" type="main" xml:lang="en">Efficient BackProp</title><author><name sortKey="Lecun, Yann" sort="Lecun, Yann" uniqKey="Lecun Y" first="Yann" last="Lecun">Yann Lecun</name><affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country><wicri:regionArea>Image Processing Research Department AT&T Labs - Research, 100 Schulz Drive, 07701-7033, RedBank, NJ</wicri:regionArea><placeName><region type="state">New Jersey</region></placeName></affiliation><affiliation><wicri:noCountry code="no comma">E-mail: yann@research.att.com</wicri:noCountry></affiliation></author><author><name sortKey="Bottou, Leon" sort="Bottou, Leon" uniqKey="Bottou L" first="Leon" last="Bottou">Leon Bottou</name><affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country><wicri:regionArea>Image Processing Research Department AT&T Labs - Research, 100 Schulz Drive, 07701-7033, RedBank, NJ</wicri:regionArea><placeName><region type="state">New Jersey</region></placeName></affiliation><affiliation><wicri:noCountry code="no comma">E-mail: leonb@research.att.com</wicri:noCountry></affiliation></author><author><name sortKey="Orr, B" sort="Orr, B" uniqKey="Orr B" first="B." last="Orr">B. Orr</name><affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country><wicri:regionArea>Willamette University, 900 State Street, 97301, Salem, OR</wicri:regionArea><placeName><region type="state">Oregon</region></placeName></affiliation><affiliation wicri:level="1"><country wicri:rule="url">États-Unis</country></affiliation></author><author><name sortKey="Muller, Robert" sort="Muller, Robert" uniqKey="Muller " first="-Robert" last="Müller">-Robert Müller</name><affiliation wicri:level="3"><country xml:lang="fr">Allemagne</country><wicri:regionArea>GMD FIRST, Rudower Chaussee 5, 12489, Berlin</wicri:regionArea><placeName><region type="land" nuts="3">Berlin</region><settlement type="city">Berlin</settlement></placeName></affiliation><affiliation wicri:level="1"><country wicri:rule="url">Allemagne</country></affiliation></author></analytic><monogr></monogr><series><title level="s">Lecture Notes in Computer Science</title><imprint><date>1998</date></imprint><idno type="ISSN">0302-9743</idno><idno type="ISSN">0302-9743</idno></series><idno type="istex">ACCB9E0CBDBB8AEDBAF9A8372F13515E36000C20</idno><idno type="DOI">10.1007/3-540-49430-8_2</idno><idno type="ChapterID">2</idno><idno type="ChapterID">Chap2</idno></biblStruct></sourceDesc><seriesStmt><idno type="ISSN">0302-9743</idno></seriesStmt></fileDesc><profileDesc><textClass></textClass><langUsage><language ident="en">en</language></langUsage></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Abstract: The convergence of back-propagation learning is analyzed so as to explain common phenomenon observedb y practitioners. Many undesirable behaviors of backprop can be avoided with tricks that are rarely exposedin serious technical publications. This paper gives some of those tricks, ando.ers explanations of why they work. Many authors have suggested that second-order optimization methods are advantageous for neural net training. It is shown that most “classical” second-order methods are impractical for large neural networks. A few methods are proposed that do not have these limitations.</div></front></TEI><affiliations><list><country><li>Allemagne</li><li>États-Unis</li></country><region><li>Berlin</li><li>New Jersey</li><li>Oregon</li></region><settlement><li>Berlin</li></settlement></list><tree><country name="États-Unis"><region name="New Jersey"><name sortKey="Lecun, Yann" sort="Lecun, Yann" uniqKey="Lecun Y" first="Yann" last="Lecun">Yann Lecun</name></region><name sortKey="Bottou, Leon" sort="Bottou, Leon" uniqKey="Bottou L" first="Leon" last="Bottou">Leon Bottou</name><name sortKey="Orr, B" sort="Orr, B" uniqKey="Orr B" first="B." last="Orr">B. Orr</name><name sortKey="Orr, B" sort="Orr, B" uniqKey="Orr B" first="B." last="Orr">B. Orr</name></country><country name="Allemagne"><region name="Berlin"><name sortKey="Muller, Robert" sort="Muller, Robert" uniqKey="Muller " first="-Robert" last="Müller">-Robert Müller</name></region><name sortKey="Muller, Robert" sort="Muller, Robert" uniqKey="Muller " first="-Robert" last="Müller">-Robert Müller</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/OcrV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 002240 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 002240 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= OcrV1
|flux= Main
|étape= Exploration
|type= RBID
|clé= ISTEX:ACCB9E0CBDBB8AEDBAF9A8372F13515E36000C20
|texte= Efficient BackProp
}}
| This area was generated with Dilib version V0.6.32. |  |